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(54) Symbol displaying apparatus 

(57) A symbol displaying apparatus comprises a flat shallow display and means for determining the path of 
a writing implement 6 on a surface 4 above the display. The display displays the path of the .mplement 6 
directly below the path on the surface 4 so as to resemble writing with a pen or pencil. When a character 
' or symbol has been recognised, its path is replaced by a formalised image. The apparatus may be used to 
enter text into a word processor and display the entered text. The apparatus may recognise alphanumeric, 
shorthand or Japanese characters, mathematical symbols, or symbols of circuit components in a circuit 
diagram. Different colours may be input using a graticule 5 electronically switched to different displayed 
colours corresponding to the input colours. 
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SPECIFICATION 
Display apparatus 

5 The present invention relates to a display 
apparatus for displaying hand written symbols, 
such as alpha numeric characters. 

In general, personal computers include three 
identifiable discrete components, namely a 
10 visual output device such as a display screen, 
a manual input device which is generally a 
keyboard, and a housing for the processing 
and storage components of the computer. 
Many personal computers are used for hand- 
15 ling text, generally by means of word process- 
ing software. However, the use of such per- 
sonal computers for informal handling of text 
causes problems, especially to untrained per- 
sonnel, since the ability to enter and manipu- 
20 late text with speed and precision requires a 
high degree of typing skill. 

For handling text, conventional computers 
have the disadvantage that input to the com- 
puter is in serial form. For the majority of 
25 applications, text input is provided by means 
of the keyboard so that the computer sees a 
serial string of numbers representing the de- 
sired alpha numeric characters. The practical 
consequence of this is demonstrated by the 
30 need for a cursor in order to indicate to the 
"computer where the next character is to be 
placed. Also, an additional input device is re- 
quired for graphics applications. 
Another problem is that the input and out- 
35 put devices are physically separated. Thus, 
when entering text via a keyboard, there is a 
constant change of attention point between 
the keyboard and the display, which is fatigu- 
ing and limits efficiency. Because of the separ- 
40 ation. such computers also occupy a compara- 
tivel y large working space. 

Much work is presently going into speech 
recognition/synthesis techniques for com- 
puters and, when developed, such' systems 
45 would avoid the problems of large working 
space requirement and change of attention po- 
int mentioned above. However, input by 
speech recognition, would still represent a se- 
rial input to the computer, with the disadvan- 
50 tages mentioned above. Also, there will be 
situations in which speech recognition is not 
practical, for instance in an office full of peo- 
ple all trying to talk to their computers. 
Light pens have been used to permit some 
55 degree of interaction between a user and a 
computer. A light pen is a device which coop- 
erates with a raster-scan display to enable in- 
putting of graphical information which is dis- 
played on a visual display unit as the light pen 
60 moves over its surface. Light pens are also 
used to perform command and control func- 
tions, for instance selecting commands dis- 
played on a screen, moving graphics and text 
from one position on a. screen to another, etc. 
65 However, light pens have certain disadvan- 


tages because of the mode of operation. 
Speed of movement is limited if data is to be 
entered reliably because of the finite scanning 
speed on conventional displays. Also, resolu- 
70 tion is limited because of the line-scanning 
* and is generally less than one picture element 
(pixel). Thus, light pens and other position de- 
termining systems dependent on the display 
scanning process are unsuitable for inputting 
75 of conventional hand-written symbols. 

According to the invention, there is provided 
an apparatus for displaying a symbol, compris- 
ing a display, first means independent of the 
display for determining the path of a writing 
80 implement on the surface of the display, sec- 
ond means for causing the display to display 
the path of the writing implement on the sur- 
face of the display in registration with the 
path of the implement, and third means for 
85 detecting when the path corresponds to a 

predetermined symbol and for causing the dis- 
play to display the predetermined symbol in \ 
place of the path. 
The display is preferably a flat display ie 
90 with a thickness which is much smaller than 
the height and width of the surface providing 
the display. Thus, an outline similar to a book 
or writing pad may be achieved. The display 
itself may comprise a liquid crystal display, 
95 plasma display, or any other suitable display. 
A liquid crystal display has the advantage that 
no potentially or allegedly harmful radiation is 
emitted. Again, much work is currentty being 
done on producing flat-screen displays and 
100 any display capable of displaying the path of 
the implement may be used. 

The position determining means may deter- 
mine physically the position of the implement 
on the display surface or screen. The position 
105 determining means may also be capable of 
determining the position of the implement on 
a surface outside the screen surface, for in- 
stance to permit the digitization of existing 
drawings. Techniques which rely on electro- 
110 magnetic or acoustic signals, pressure, and 
the like may be used. 

The writing implement need not be a part of 
the apparatus and may be a conventional pen, 
preferably a pen which does not dispense ink 
115 since the display itself shows the path fol- 
lowed by the implement. With suitable posi- 
tion determining means, the implement could 
even be a human finger. 
The third means preferably includes charac- 
120 ter recognition means for recognising alpha 
numeric characters. Other characters, such as 
shorthand or Japanese characters or mathe- 
matical symbols could also be made recognis- 
able by the use of the appropriate character 
125 recognition means. Such means may operate 
in any suitable way. for instance on the basis 
of the shape of the path or on the basis of 
the strokes used to form the path. 
The apparatus may form part of a computer 
130 including a processing unit and storage. Such 
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a computer may be used for word processing 
or other handling of text, or even simply for 
storage of text. In a preferred embodiment, 
the writing implement is used in the way of a 
5 conventional pen or pencil to "write" on the 
display surface or screen of the display. The 
display produces a visual output of the path 
followed by the implement, just as though the 
implement were a conventional pen leaving a 
10 trail of ink on paper. At a suitable opportunity, 
for instance after recognition of each character 
or of each word (each blank space) or at the 
end of each line, recognised characters are 
substituted on the display in place of the ac- 
15 tual path of the implement. Such characters 
may be in the form of. a conventional dot 
matrix representation of the Roman alphabet 
and Arabic-based numbers in common usage 
throughout the world. However, for particular 
20 applications, different character recognition 
means and different "fonts" of characters or 
graphic symbols may be used. 

The apparatus may be arranged to scale the 
recognised symbols automatically. Thus, when 
25 data characters are recognised, the replace- 
ment symbol may be the same size as the 
hand-written character or may be taken from a 
set of predetermined sizes. 
Any suitable recongition means may be 
30 used. Although present character recognition 
means cannot adequately handle connected 
writing but require the input of text in sepa- 
rate characters, future developments should 
improve the recognition abilities with con- 
35 nected writing. 

Another advantage of such an embodiment 
is that text and graphics may be mixed. The 
conventional form of input corresponding di- 
rectly to the use of pen and paper avoids the 
40 problem of serial input of symbols and allows 
graphic symbols, mathematical formulae, and 

e^eTTConventional drawings to be mixed with 

text in the same way as is possible with con- 
ventional pen and paper. For entering technical 
45 drawings, such as circuit diagrams, the recog- 
nition means could be arranged to recognise 
conventional symbols, for instance for transis- 
tors, resistors, and other components and to 
substitute standard symbolic representations 
50 of these so as to produce a neat circuit dia- 
gram. The recognition means is preferabty ar- 
ranged to allow control and command func- 
tions to be recognised and performed, such 
as erasure of symbols, moving of symbols 
55 {"picking" and "dragging"), and selection 
from a command menu. 

An apparatus of this type removes the need 
for conventional typing skills and makes com- 
puting more accessible to untrained personnel. 
60 problems with change of attention point are 
avoided because the apparatus can simulate 
very closely conventional writing by means of 
a pen or pencil with the input and output oc- 
curring at essentially the same point in space. 
65 Also, by combining input and output devices, 


the space requirement can be substantially re- 
duced. 

Such an apparatus has many applications 
and some examples of such applications are: 

70 automatic translation of shorthand into text; 
access to personal computers for non-euro- 
pean language users, for instance Japanese 
users, in their own language; programming 
and representation of information using graphi- 

75 cal and/or mathematical symbols; and teach- 
ing of handwriting to children. 

A dim graticule may be displayed on the 
display in order to guide writing by means of 
the implement. In order to provide multi-colour 

80 input, an electronically switched graticule may 
be used to select the input colour. For in- 
stance, each graticule may be coloured in ac- 
cordance with the colour which can be input 
when that graticule is selected. This technique 

85 again is very similar to conventional use of 
different coloured pens or pencils, where the 
pen or pencil has to be exchanged for another 
when it is desired to change colour. This 
represents a consideraole advantage and is no 

90 less convenient than other forms of "pallet 
selection" used for multi-colour graphics with 
conventional computers. 

The character recognition means may be of 
a programmable or "trainable" type, in which, 

95 for instance, a symbol is drawn once or sev- 
eral times and the character recognition means 
is informed of which corresponding "formal- 
ised" symbol corresponds thereto. Once this 
basic training or programming has been fin- 
100 ished, drawing of the symbol on the display 
by means of the implement wiii thereafter re- 
sult in the path of the implement on the dis- 
play being replaced by the formalised symbol. 
The invention will be further described, by 
105 way of example, with reference to the accom- 
panying drawings, in which: 

Figure 1 shows a display apparatus consti- 
tuting a preferred embodiment of the inven- 
tion; 

1 10 Figure 2 is a cross-sectional view of the 
apparatus of Figure 1 ; 

Figure 3 is a functional block diagram of the 
apparatus of Figure 1; 
Figure 4 is a more detailed block diagram of 
115 the apparatus of Figure 1 ; 

Figure 5 shows a writing implement of the 
apparatus of Figure 1; 

Figure 6 is a cross-sectional view of pan of 
the implement of Figure 5; 
120 Figure 7 shows a detail of Figure 6 to an 
enlarged scale; 

Figure 8 is a block diagram of part of the 
apparatus of Figure 1; 
Figures 9A and 9B are a flow diagram illus- 
125 trating operation of the apparatus of Figure 1; 
Figure 10 illustrates a possible screen lay- 
out; 

Figure 1 1 illustrates some control character 
operations; and 
130 Figure 12 is a flow diagram illustrating oper- 
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ation of a character recognition system learn- 
ing phase. 

The display apparatus shown in Figures 1 
and 2 comprises a wedge-shaped housing 1 

5 with rear feet 2 which may be adjusted for 
height. Most of the sloping major surface of 
the housing is taken up by a large flat display 
3, for instance of LCD type, covered by a thin 
sheet of protective glass 4. The surface of the 

10 sheet 4 has a thin transparent conductive film 
which is electrically connected to earth, and a 
faint switchable graticule 5. A pen 6 and sen- 
sors 8 and 9 are provided for a position de- 
termining system for determining the position 

15 of the pen on the surface of the sheet 4. A 
slot 7 is provided for housing the pen when 
not in use. 

The sloping face also carries function swit- 
ches 10, 11, a removable indicator strip 12 
20 for the function switches, cartridge slots 13 
for insertion of external cartridges holding pre- 
prepared programs, a loudspeaker 15 for a 
speech synthesiser, and light emitting diode 
indicators 16. The side of the housing has a 
25 disc drive 14 for mass storage and a power 
switch 17. The front edge of the housing car- 
ries a connector 18 for an external keyboard, 
which may more conveniently be used by 
touch typists who can type faster than they 
30 can write. 

Figure 2 illustrates the internal construction 
of the apparatus. The electronic circuitry is 
carried by one or more boards 19 fixed to the 
bottom of the housing by supports 20. A re- 
35 cess 21 is provided at the rear of the housing 
and contains various external connectors 22. 

As shown in Figure 3, the apparatus com- 
prises a processing system 31 which contains 
a character recognition system 32. The pro- 
40 cessing system 31 has various interfaces in- 
cluding a serial input/output port 33, a car- 
tridge controller 34 for magnetic tape car- 
tridges 24 or the like, a mass storage inter- 
face 35 for the disc drive 14, a speech syn- 
45 thesiser 36 for the loudspeaker 15, a display 
driver 37 for the display 3 and for an external 
monitor line 38, a coordinate digitizing inter- 
face 39 for the pen 6, and a switch and key- 
board interface 40 for an external keyboard 
50 23 if required. With the exception of the co- 
ordinate digitizing interface 39, the other parts 
are generally conventional and frequently pro- 
vided in personal computers. Figure 4 illus- 
trates in more detail the hardware which con- 
55 stitutes the apparatus shown in Figure 3. 
The pen 6 has a body which contains a 
switch 130 and a tip 131 which acts as a 
point source of ultrasonic radiation. The ultra- 
sonic wave front emitted by the tip is re- 
60 ceived by the sensors 8 and 9 and. by mea- 
suring the time delay between transmission 
and reception of an ultrasonic burst, the pen 
position may be determined as indicated here- 
inafter. The tip 131 is. made of metal and is 
65 earthed by the coating on the glass sheet 4 


so as to determine when the pen touches the 
display surface. An ultrasonic transmitter 132 
surrounded by absorptive rubber 133 commu- 
nicates via a channel 134 with the tip 131. 
70 As shown in Figure 7, the tip 131 has a small 
convexly tapered reflector 135 for reflecting 
the ultrasonic signals from the transducer 132 
generally transversely of the pen and in all 
directions. The tip 135 is fixed to the body 
75 by means of thin supports 136 and a con- 
necting tag 137. The diameter of the tip is 
approximately equal to a wavelength at the 
frequency of the ultrasonic signal. 
Figure 8 illustrates the coordinate digitizing 
80 system in more detail. This system comprises 
a timing generator 150 which supplies timing 
signals for clocking, clearing, and starting 
counting of two counters 151 and 152. The 
timing generator receives clock signals on an 
85 input line 153 from the microprocessor timing 
generator shown in Figure 4. The timing gen- 
erator 150 also supplies an ultrasonic signal 
via a bandpass filter 154 to the pen 6 via a 
control circuit comprising an operational ampli- 
90 fier 155 resistors RA, RB, and RF, and an 
electronic switch 156 controlled by a start 
burst signal from the generator 150. Thus, the 
ultrasonic transducer transmits the ultrasonic 
signal continuously but, at the start of each 
95 digitizing step, the switch 156 is closed tem- 
porarily to provide an increased intensity or 
transient level. 

The sensors 8 and 9 are connected to am- 
plifiers 157 and 158, bandpass filters 159 
100 and 160, detectors 161 and 162. and com- 
parators 163 and 164, respectively. The sig- 
nals received by the sensors are thus pro- 
cessed and the comparators compare the en- 
velopes of the signals with a reference voltage 
105 V, i( . When the reference voltage is exceeded, 
the comparators 163 and 164 stop the coun- 
ters 151 and 152. Latching of the count is 
controlled by a latch signal from the timing 
generator 150. The tip 131 of the pen 6 is 
110 earthed when the tip is in contact with the 
transparent conductive coating on the glass 
sheet 4 and this is used to indicate pen up/- 
pen down. The counters 151 and 152 are 
each twelve bit counters and are cleared and 
1 1 5 instructed to start counting the clock pulses 
coincident with the start burst emitted by the 
transducer 132 after a delay to allow for the 
delay introduced by the operation of the band- 
pass filters 159, 160 and detectors 161, 162. 
120 The counters are stopped upon detection of 
the start burst signals by the sensors 8 and 9 
so that the counts held represent the distance 
from the sensors 8 and 9 to the pen tip 131. 
In order to convert these positions to carte- 
125 sian coordinates, the microprocessor is prefer- 
ably arranged to determine the x and y coordi- 
nates from the following equations: 

x = (D1 2 - D2 2 )/4a + a 
130 (D1 2 - x 2 )" 2 
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where D1 and D2 are the counts in the coun- 
ters 151 and 152 and a is half the spacing 
between the sensors 8 and 9. 
5 Figures 9A and 9B illustrate operation of the 
display apparatus. Upon switching on the 
apparatus, following certain initialising and 
housekeeping functions, the apparatus pro- 
ceeds from the start block 200. Block 201 
10 determines whether a character has been 
started and, if not, loops back until a charac- 
ter has been started. Block 202 determines 
the position of the pen tip as described here- 
inbefore and stores this in display memory so 
15 that the display displays a path following the 
movements of the pen tip. The apparatus then 
enters a character recognition routine. 

In the character recognition routine, move- 
ments of the pen tip are separated into indivr- 
20 dual strokes at block 203. Until the end of a 
character is detected in block 204, operation 
cycles back to block 202. Upon detection of 
the end of a character, the salient features 
defining the character are extracted in block 
25 205 and a dictionary of characters is searched 
in block 206. If the search is successful and 
the character is recognised, the operation 
moves on. If not, the pen trace on the display 
is replaced by an indication that an unrecog- 
30 nised character has been written in block 208, 
the unrecognised character is placed into a 
buffer in block 209 and control returns to 
block 201 for the next character. 
When a character is recognised, block 210 
35 determines whether the character is a data 
character or some other character, such as a 
control character. If the character is a control 
character, block 211 determines whether it is 
a valid control character and if not informs the 
40 user in block 212 and returns to the start 
block 200. If a valid character is detected, the 
desired control operation is performed in block 
213 and the pen trace is removed from the 
screen in block 214. 
45 If a data character is recognised, block 215 
determines whether the character is in a com- 
mand window as will be described hereinafter. 
If not, the character written by the pen is 
replaced, in block 216, with a formalised dis- 
50 play of the character, the character is placed 
in a data buffer 217, and the contents of the 
buffer are passed in block 218 for further pro- 
cessing, after which the operation returns to 
the start block 200. 
55 f If the data character occurs in a com—and 
window, the pen trace is replaced by a for- 
malised display of the character in block 219 
and the character is placed in a command 
buffer in block 220. Block 221 determines 
60 whether the character is a correct valid com- 
mand or part thereof, and, if not, the user is 
informed at block 222 and control returns to 
the start block 200. If valid, block 223 deter- 
mines whether the command is complete. If 
65 not, control is passed to the start block 200. 


If the command is complete, the desired oper- 
ation is performed at block 224 and control 
returns to the start block 200. 
Figure 10 illustrates a typical screen display 

70 partioning. The screen has a command area 
325 in which characters may be written by 
means of the pen for causing certain com- 
mands to be performed. The four corners 326 
of the display are allocated to scroll functions 

75 in the directions indicated by the arrows and 
placing the pen tip in any of these corners will 
cause corresponding scrolling of the display. 
A status area 327 is set aside for allowing 
the apparatus to display status and other in- 

80 formation to the user, such as errors, placing 
the pen in the status area normally has no 
effect. A data region 328 is provided for writ- 
ing characters for data or control by means of 
the pen. One or more menu areas 329 may 

85 be provided in the command area 325 to al- 
low such commands to be executed instantly 
by placing the pen tip at the relevant menu 
area. 

In order to allow free-hand graphics to be 
90 entered, the apparatus may be arranged, upon 
receipt of a command written in the command 
window or upon actuation of a programmable 
function switch, to disable character recogni- 
tion. In this mode, the apparatus functions in 
95 much the same way as a conventional graphic 
system, for instance based on "picking" and 
"dragging". 

Figure 1 1 illustrates various control charac- 
ters which control the display of data. Thus, 

100 rapid horizontal movements erase a character 
and permit insertion of a fresh character. En- 
circling of one or more previous characters 
allows a block operation to be executed, such 
as move, delete, and the like. Rapid vertical 

105 movements followed by a path crossing previ- 
ous characters causes a text line to be de- 
leted. Touching a character and performing a 
curved path causes the character to be ro- 
tated. 

1 10 Prior to normal operation of the apparatus, 
the character recognition system may be 
trained to recognise characters written by a 
particular user. Figure 12 illustrates a method 
of performing such training. Following a start 

1 15 block 400, a user enters a character represen- 
tation code sequence for selecting a specific 
character in block 401. For instance, the AS- 
CII code may be entered to select a symbol 
which is to be programmed. The user may 

120 then produce a desired replacement graphic 
display from an inbuilt graphic editor in block 
402. The user then provides a number of 
hand drawn examples of the character by 
means of the pen on the display at block 

125 403. The features of the hand drawn 

examples are extracted and, in block 404, a 
test is performed to assess whether these 
features clash or conflict with those already 
existing. If so, control returns to the block 

130 403 for another attempt. If not, the features 
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are stored in a dictionary in block 405 and 
the graphic and character representations are 
stored in block 406. This routine may be re- 
peated for all characters which are required to 

5 be recognised. 

Various modifications may be made within 
the scope of the invention. For instance, the 
ultrasonic means for digitizing the position of 
the pen on the display may be replaced by 

10 alternative systems. One example is a spark 
generator provided at the tip of the pen and 
controlled by the timing generator. 

CLAIMS 

15 1. An apparatus for displaying a symbol, 
comprising a display, first means independent 
of the display for determining the path of a 
writing implement on the surface of the dis- 
play, second means for causing the display to 

20 display the path of the writing implement on 
the surface of the display in registration with 
the path of the implement, and third means 
for detecting when the path corresponds to a 
predetermined symbol and for causing the dis- 

25 play to display the predetermined symbol in 
place of the path. 

2. An apparatus as claimed in Claim 1, in 
which the display is substantially flat and has 
a thickness substantially less than the height 

30 and width thereof. 

3. An apparatus as claimed in Claim 1 or 2, 
in which the first means is arranged selectively 
to determine the position of a writing imple- 
ment on a surface different from the surface 

35 of the display. 

4. An apparatus as claimed in any one of 
the preceding claims, in which thi* first means 
comprises sound producing or detecting 
means arranged to be disposed at the tip of a 

40 writing implement, at least two sound detect- 
ing or producing means, and processing 
means for determining the position of the tip 
of the writing implement from the propagation 
times of sound from the sound producing 

45 means. 

5. An apparatus as claimed in any one of 
the preceding claims, in which the third means 
comprises character recognition means for se- 
lecting from a predetermined set of alphanu- 

50 meric characters and/or symbols a character 
or symbol which most closely resembles the 
path. 

6. An apparatus as claimed in claim 5, in 
which the third means is arranged to cause 

55 the display to display the selected character 
or symbol in place of the path. 

7. An apparatus as claimed in claim 5, in 
which the third means is arranged to cause 
the display to display a character or symbol 

60 different from but associated with the selected 
character or symbol in place of the paih. 

8. An apparatus as claimed in Claim 5, in 
which the character recognition means is ar- 
ranged to recognise a path representing a 

65 command and to provide a signal for control- 


ling implementation of the command. 

9. An apparatus substantially as hereinbe- 
fore described with reference to and as illus- 
trated in the accompanying drawings. 
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